Graph Theory Approaches for Optimizing Biomedical Data Analysis Using Reproducible Workflows

ly, computational workflows may be understood as a directed acyclic graph (DAG), a kind of finite graph which contains no cycles and which must be traversed in a specific direction. In this representation, each node is an individual executable command. The edges in the DAG represent execution variables (data elements such as files or parameters) which pass from upstream nodes to downstream ones. Figure 1. Illustration of a directed acyclic graph (DAG). The DAG may be traversed from left-to-right, moving from node-to-node along the edges that connect them. In practice, workflows are described with machine-readable serialized data objects in either a general-purpose programming language (GPL), domain-specific language (DSL), or serialized object models for workflow description. For example, an object model-based approach may describe the steps in a workflow in JSON format with a custom syntax. This workflow description can then be parsed by an engine or executor to create the DAG representation of the workflow. The executor may then translate the directions for workflow execution to actionable jobs in which data is analyzed on a computational infrastructure or backend. 3.1. General structure of a workflow execution There are three general steps in workflow execution: interpretation of a machine-readable workflow description, generation of the workflow DAG, and finally decomposition into individual jobs that can be scheduled for execution. At the beginning of execution, a workflow engine or interpreter is provided with the workflow description and the required inputs for execution of the workflow, such as parameters and file paths (Fig. 2a). The workflow description object is then parsed and a DAG is created (Fig. 2b), which contains the most minimal set of nodes and edges required for computation. In addition to representing the steps in the workflow as a DAG (Fig. 2c), many workflow ontologies model computational jobs as a composite (tree) pattern in which there are “parent nodes” (workflows), which can contain multiple executables or “leaf nodes” (executables) (Fig. 2d). The Rabix engine extends this model by generalizing “parent” nodes to include transformations of the DAG, such as when parallelizations are possible at that node. The engine handles the "execution" or parsing of these parent jobs, while leaves are queued for scheduling and execution on a backend. This model allows for more efficient resolution of DAG features such as . CC-BY-NC-ND 4.0 International license peer-reviewed) is the author/funder. It is made available under a The copyright holder for this preprint (which was not . http://dx.doi.org/10.1101/074708 doi: bioRxiv preprint first posted online Sep. 12, 2016;